Structured Models from Structured Data: Emergence of Modular Information Processing within One Sheet of Neurons

In our contribution we investigate how structured information processing within a neural net can emerge as a result of unsupervised learning from data. Our model consists of input neurons and hidden neurons which are recurrently connected and which represent the thalamus and the cortex, respectively. On the basis of a maximum likelihood framework the task is to generate given input data using the code of the hidden units. Hidden neurons are fully connected allowing for diierent roles to play within the unfolding time-dynamics of this data generation process. One parameter which is related to the sparsity of neuronal activation varies across the hidden neurons. As a result of training the net captures the structure of the data generation process. Trained on data which are generated by diierent mechanisms acting in parallel, the more active neurons will code for the more frequent input features. Trained on hierarchically generated data, the more active neurons will code on the higher level where each feature integrates several lower level features. The results imply that the division of the cortex into laterally and hierarchically organized areas can evolve to a certain degree as an adaptation to the environment.